14 |
A Practical Methodology for the Evaluation of Spoken Language Systems
|
|
|
|
In: DTIC (1992)
|
|
Abstract:
A meaningful evaluation methodology can advance the state-of-the-art by encouraging mature, practical applications rather than "toy" implementations. Evaluation is also crucial to assessing competing claims and identifying promising technical approaches. While work in speech recognition (SR) has a history of evaluation methodologies that permit comparison among various systems, until recently no methodology existed for either developers of natural language (NL) interfaces or researchers in speech understanding (SU) to evaluate and compare the systems they developed. Recently considerable progress has been made by a number of groups involved in the DARPA Spoken Language Systems (SLS) program to agree on a methodology for comparative evaluation of SLS systems, and that methodology has been put into practice several times in comparative tests of several SLS systems. These evaluations are probably the only NL evaluations other than the series of Message Understanding Conferences (Sundheim, 1989; Sundheim, 1991) to have been developed and used by a group of researchers at different sites, although several excellent workshops have been held to study some of these problems (Palmer et al., 1989; Neal et a!., 1991). ; Supported in part by DARPA.
|
|
Keyword:
*NATURAL LANGUAGE; *SPEECH RECOGNITION; HISTORY; LANGUAGE; Linguistics; MESSAGE PROCESSING; METHODOLOGY; SPEECH; TEST AND EVALUATION; WORKSHOPS
|
|
URL: http://www.dtic.mil/docs/citations/ADA457494 http://oai.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA457494
|
|
BASE
|
|
Hide details
|
|
15 |
BBN BYBLOS and HARC February 1992 ATIS Benchmark Results
|
|
|
|
In: DTIC (1992)
|
|
BASE
|
|
Show details
|
|
16 |
Using Spoken Language to Facilitate Military Transportation Planning
|
|
|
|
In: DTIC (1991)
|
|
BASE
|
|
Show details
|
|
17 |
A Proposal for Incremental Dialogue Evaluation
|
|
|
|
In: DTIC (1991)
|
|
BASE
|
|
Show details
|
|
18 |
Research and Development in Natural Language Understanding as Part of the Strategic Computing Program. Volume 1. Overview of Technical Results. Revision.
|
|
|
|
In: DTIC AND NTIS (1990)
|
|
BASE
|
|
Show details
|
|
19 |
Research and Development in Natural Language Understanding as Part of the Strategic Computing Program
|
|
|
|
In: DTIC AND NTIS (1988)
|
|
BASE
|
|
Show details
|
|
20 |
Requirements for Natural Language Understanding in a System with Graphic Displays
|
|
|
|
In: DTIC AND NTIS (1983)
|
|
BASE
|
|
Show details
|
|
|
|